Determining a number of unique incidents in a plurality of incidents for incident processing in a distributed processing system

ABSTRACT

Methods, apparatuses, and computer program products for determining a number of unique incidents in a plurality of incidents for incident processing in a distributed processing system are provided. Embodiments include an incident analyzer identifying within the plurality of incidents, attribute combination entries of location identifications and incident types and analyzing each location identification in each attribute combination entry according to a sequence of the attribute combination entries including creating attribute pairs. The incident analyzer is also configured to count the attribute pairs. The number of attribute pairs is the number of unique incidents in the plurality of incidents.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The field of the invention is data processing, or, more specifically,methods, apparatuses, and computer program products for determining anumber of unique incidents in a plurality of incidents for incidentprocessing in a distributed processing system.

2. Description of Related Art

The development of the EDVAC computer system of 1948 is often cited asthe beginning of the computer era. Since that time, computer systemshave evolved into extremely complicated devices. Today's computers aremuch more sophisticated than early systems such as the EDVAC. Computersystems typically include a combination of hardware and softwarecomponents, application programs, operating systems, processors, buses,memory, input/output devices, and so on. As advances in semiconductorprocessing and computer architecture push the performance of thecomputer higher and higher, more sophisticated computer software hasevolved to take advantage of the higher performance of the hardware,resulting in computer systems today that are much more powerful thanjust a few years ago.

Modern distributed processing systems for intensive computing may havemillions of devices with many processes running on each device all ofwhich are capable of error and status reporting for automated errorrecovery, reporting to a systems administrator, and for other reasons.In many cases, in the case of an error for example, the sheer number ofsuch error reports and status reports are so overwhelming that theycannot be handled in a meaningful manner. For example, a systemsadministrator receiving a hundred thousand error reports may beoverwhelmed by the sheer number of such reports and therefore in theaggregate those reports become more and more unhelpful and irrelevant.

SUMMARY OF THE INVENTION

Methods, apparatuses, and computer program products for determining anumber of unique incidents in a plurality of incidents for incidentprocessing in a distributed processing system are provided. Embodimentsinclude an incident analyzer identifying within the plurality ofincidents, attribute combination entries of location identifications andincident types and analyzing each location identification in eachattribute combination entry according to a sequence of the attributecombination entries including creating attribute pairs. The incidentanalyzer is also configured to count the attribute pairs. The number ofattribute pairs is the number of unique incidents in the plurality ofincidents.

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescriptions of exemplary embodiments of the invention as illustrated inthe accompanying drawings wherein like reference numbers generallyrepresent like parts of exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system for determining a number ofunique incidents in a plurality of incidents for incident processing ina distributed processing system according to embodiments of the presentinvention.

FIG. 2 sets forth a block diagram of automated computing machinerycomprising an exemplary computer useful in determining a number ofunique incidents in a plurality of incidents for incident processing ina distributed processing system according to embodiments of the presentinvention.

FIG. 3 sets forth a block diagram of an exemplary system for determininga number of unique incidents in a plurality of incidents for incidentprocessing in a distributed processing system according to embodimentsof the present invention.

FIG. 4 sets forth a diagram illustrating assigning events to an eventspool according to embodiments of the present invention.

FIG. 5 sets forth a diagram illustrating assigning alerts to an alertspool according to embodiments of the present invention.

FIG. 6 sets forth a flow chart illustrating an example method ofadministering incident pools for incident processing in a distributedprocessing system according to embodiments of the present invention.

FIG. 7 sets forth a flow chart illustrating an additional method ofdetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention.

FIG. 8 sets forth a flow chart illustrating an additional method ofdetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention.

FIG. 9 sets forth a flow chart illustrating an additional method ofdetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention.

FIG. 10 sets forth a flow chart illustrating an additional method ofdetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary methods, apparatuses, and computer program products fordetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention are described with reference to theaccompanying drawings, beginning with FIG. 1. FIG. 1 illustrates anexemplary system for determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system (101) according to embodiments of the presentinvention. A distributed processing system is typically implemented asmultiple autonomous or semi-autonomous computers that communicatethrough a computer network. In such example distributed processingsystems, the computers often interact with each other in order toachieve a common goal. A computer program that runs in such an exampledistributed system is typically called a distributed program, anddistributed programming is often used to describe the process of writingsuch programs.

In the example of FIG. 1, the distributed processing system (101) isimplemented as a parallel computer (100), non-volatile memory for thecomputer in the form of data storage device (118), an output device forthe computer in the form of a printer (120), and an input/output devicefor the computer in the form of a computer terminal (122). The parallelcomputer (100) in the example of FIG. 1 includes a plurality of computenodes (102). Each compute node is an automated computing device composedof one or more computer processors, its own computer memory, and its owninput/output functionality. The compute nodes (102) are coupled for datacommunications by several independent data communications networksincluding a high speed Ethernet network (174), a Joint Test Action Group(‘JTAG’) network (104), a collective or tree network (106) which isoptimized for collective operations, and a torus network (108) which isoptimized for point to point operations. The tree network (106) is adata communications network that includes data communications linksconnected to the compute nodes so as to organize the compute nodes as atree. Each data communications network is implemented with datacommunications links among the compute nodes (102). The datacommunications links provide data communications for parallel operationsamong the compute nodes of the parallel computer (100).

In addition to the compute nodes (102), the parallel computer (100)includes input/output (‘I/O’) nodes (110, 114) coupled to the computenodes (102) through the high speed Ethernet network (174). The I/O nodes(110, 114) provide I/O services between the compute nodes (102) and I/Odevices, which in this example is the data storage device (118), theprinter (120) and the terminal (122). The I/O nodes (110, 114) areconnected for data communications through a local area network (‘LAN’)(130). The parallel computer (100) also includes a service node (116)coupled to the compute nodes (102) through the JTAG network (104). Theservice node (116) provides service common to the compute nodes (102),such as loading programs into the compute nodes (102), starting programexecution on the compute nodes (102), retrieving results of programoperations on the compute nodes (102), and so on. The service node (116)runs an event and alert analysis module (124) and communicates with asystem administrator (128) through a service application interface (126)that runs on the computer terminal (122).

Many of the components of the distributed processing system of FIG. 1,that is the devices of the distributed processing system or processesrunning on the devices of the distributed processing system of FIG. 1,are capable of some form of error or status reporting through events andmany of such components are also capable of receiving alerts in responseto one or more of such events. Often in distributed processing systemshundreds of thousands or millions of components may provide or receiveincidents, often in the form of events or alerts.

An incident is a generic term used in this specification to mean anidentification or notification of a particular occurrence on a componentof a distributed processing system such as events described below, arefined identification of an occurrence often based on events such as analert described below, or other notifications as will occur to those ofskill in the art.

Incidents are administered in pools for event and alert analysisaccording to embodiments of the present invention. A pool of incidentsis a collection of incidents organized by the time of either theiroccurrence, by the time they are logged in an incident queue, includedin the pool, or other time as will occur to those of skill in the art.Such incident pools often provide the ability to analyze a group of timerelated incidents. Often such incident pools are useful in identifyingfewer and more relevant incidents in dependence upon multiple relatedincidents.

An event according to embodiments of the present invention is anotification of a particular occurrence in or on a component of thedistributed processing system. Such events are sent from the componentupon which the occurrence occurred or another reporting component to anevent and alert analysis module according to the present invention.Often events are notifications of errors occurring in a component of thedata processing system. Events are often implemented as messages eithersent through a data communications network or shared memory. Typicalevents for event and alert analysis according to embodiments of thepresent invention include attributes such as an occurred time, a loggedtime, an event type, an event ID, a reporting component, locationinformation, and a source component, and other attributes.

An alert according to embodiments of the present invention is a refinedidentification of an occurrence, such as an error, based upon more thanone event and therefore provides an identification of the occurrence inthe context of its operation in the distributed processing system. Oftenan alert may be a notification of a particular error type of occurrencethat is identified in dependence upon the plurality of events receivedfrom one or more components of the data processing system, such as, forexample, a link failure among a plurality of devices each of which areproducing many events based upon the single link failure, or a powerfailure provoking thousands of events, and so on.

Alerts are often implemented as messages to be sent through a datacommunications network or shared memory. Typical alerts according toembodiments of the present invention have attributes attached to thembased upon the attributes of the events received from which they areidentified.

The event and alert analysis module (124) includes at least two incidentanalyzers implemented as an event analyzer and an alert analyzer capableof determining a number of unique incidents in a plurality of incidentsfor incident processing in a distributed processing system according toembodiments of the present invention. The event and alert analysismodule (124) is also implemented as a monitor and checkpoint manager formanaging the checkpoints from the incident analyzers.

Specifically, the event and alert analysis module (124) is implementedas automated computing machinery configured to identify within aplurality of incidents, attribute combination entries of locationidentifications and incident types. Each attribute combination entry hasone location identification and a set of unique incident typescorresponding to the location identification. The event and alertanalysis module (124) is also configured to analyze each locationidentification in each attribute combination entry according to asequence of the attribute combination entries. For each locationidentification of each attribute combination entry, the event and alertanalysis module (124) is configured to determine whether the set ofunique incident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair. If the set of unique incident typesincludes an incident type not selected for pairing with another locationidentification in an attribute pair, the event and alert analysis module(124) is configured to select the incident type for pairing with thelocation identification in an attribute pair and to create an attributepair of the selected incident type and the location identification. Theevent and alert analysis module (124) is also configured to count theattribute pairs, where the number of attribute pairs is the number ofunique incidents in the plurality of incidents.

The arrangement of nodes, networks, and I/O devices making up theexemplary distributed processing system illustrated in FIG. 1 are forexplanation only, not for limitation of the present invention.Distributed data processing systems configured to determine a number ofunique incidents in a plurality of incidents for incident processingaccording to embodiments of the present invention may include additionalnodes, networks, devices, and architectures, not shown in FIG. 1, aswill occur to those of skill in the art. The parallel computer (100) inthe example of FIG. 1 includes sixteen compute nodes (102). Parallelcomputers configured to determine a number of unique incidents in aplurality of incidents for incident processing according to embodimentsof the present invention sometimes include thousands of compute nodes.In addition to Ethernet, JTAG, collective, and point to point, networksin such data processing systems may support many data communicationsprotocols including for example TCP (Transmission Control Protocol), IP(Internet Protocol), and others as will occur to those of skill in theart. Various embodiments of the present invention may be implemented ona variety of hardware platforms in addition to those illustrated in FIG.1.

Determining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system in accordancewith the present invention is generally implemented with computers, thatis, with automated computing machinery. In the system of FIG. 1, forexample, all the service nodes, I/O nodes, compute nodes, of theparallel computer are implemented to some extent at least as computers.For further explanation, therefore, FIG. 2 sets forth a block diagram ofautomated computing machinery comprising an exemplary computer (252)useful in determining a number of unique incidents in a plurality ofincidents for incident processing in a distributed processing systemaccording to embodiments of the present invention. The computer (252) ofFIG. 2 includes at least one computer processor (256) or ‘CPU’ as wellas random access memory (268) (‘RAM’) which is connected through a highspeed memory bus (266) and bus adapter (258) to processor (256) and toother components of the computer (252) and through an expansion bus toadapters for communications with other components of a distributedprocessing system (101).

Stored in RAM (268) is an event and alert analysis module (124), amodule of automated computing machinery for determining a number ofunique incidents in a plurality of incidents for incident processing ina distributed processing system according to embodiments of the presentinvention. The event and alert analysis module (124) includes twoincident analyzers, a monitor (204), and a checkpoint manager (299)according to embodiments of the present invention.

The checkpoint manager (299) is configured to process checkpoints fromthe incident analyzers. In the example of FIG. 2, the monitor (204)receives events from components of the distributed processing system andputs the received events in an event queue. The monitor (204) of FIG. 2may receive events from components of the distributed processing systemon their motion, may periodically poll one or more of the components ofthe distributed processing system, or receive events from components inother ways as will occur to those of skill in the art.

The incident analyzers include an event analyzer (208) and an alertanalyzer (218). The event analyzer of FIG. 2 is a module of automatedcomputing machinery capable of identifying alerts in dependence uponreceived events. That is, event analyzers typically receive events andproduce alerts. In many embodiments, event analyzers are implemented inparallel. Often such event analyzers are assigned to a particular poolof events and may be focused on events from a particular component orcaused by a particular occurrence to produce a more concise set ofalerts.

The alert analyzer (218) of FIG. 2 is a module of automated computingmachinery capable of identifying alerts for transmission from events andother alerts, identifying additional alerts for transmission, andsuppressing unnecessary, irrelevant, or otherwise unwanted alertsidentified by the event analyzer. That is, alert analyzers typicallyreceive alerts and events and produce or forward alerts in dependenceupon those alerts and events. In many embodiments, alert analyzers areimplemented in parallel. Often such alert analyzers are assigned to aparticular pool of alerts and may be focused on alerts with particularattributes to produce a more concise set of alerts.

In addition to the general functions described above, the event andalert analysis module (124) may be configured to determine a number ofunique incidents in a plurality of incidents for incident processing ina distributed processing system according to embodiments of the presentinvention. Specifically, the event and alert analysis module (124) isimplemented as automated computing machinery configured to identifywithin a plurality of incidents, attribute combination entries oflocation identifications and incident types. Each attribute combinationentry has one location identification and a set of unique incident typescorresponding to the location identification. The event and alertanalysis module (124) is also configured to analyze each locationidentification in each attribute combination entry according to asequence of the attribute combination entries. For each locationidentification of each attribute combination entry, the event and alertanalysis module (124) is configured to determine whether the set ofunique incident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair. If the set of unique incident typesincludes an incident type not selected for pairing with another locationidentification in an attribute pair, the event and alert analysis module(124) is configured to select the incident type for pairing with thelocation identification in an attribute pair and to create an attributepair of the selected incident type and the location identification. Theevent and alert analysis module (124) is also configured to count theattribute pairs, where the number of attribute pairs is the number ofunique incidents in the plurality of incidents.

Also stored in RAM (268) is an operating system (254). Operating systemsuseful for relevant alert delivery according to embodiments of thepresent invention include UNIX™, Linux™, Microsoft XP™, AIX™, IBM'si5/OS™, and others as will occur to those of skill in the art. Theoperating system (254), event and alert analysis module (124), the eventanalyzer (208), the alert analyzer (218) in the example of FIG. 2 areshown in RAM (268), but many components of such software typically arestored in non-volatile memory also, such as, for example, on a diskdrive (270).

The computer (252) of FIG. 2 includes disk drive adapter (272) coupledthrough expansion bus (260) and bus adapter (258) to processor (256) andother components of the computer (252). The disk drive adapter (272)connects non-volatile data storage to the computer (252) in the form ofdisk drive (270). Disk drive adapters useful in computers fordetermining a number of unique incidents in a plurality of incidents forincident processing in a distributed processing system according toembodiments of the present invention include Integrated DriveElectronics (‘IDE’) adapters, Small Computer System Interface (‘SCSI’)adapters, and others as will occur to those of skill in the art.Non-volatile computer memory also may be implemented for as an opticaldisk drive, electrically erasable programmable read-only memory(so-called ‘EEPROM’ or ‘Flash’ memory), RAM drives, and so on, as willoccur to those of skill in the art.

The example computer (252) of FIG. 2 includes one or more input/output(‘I/O’) adapters (278). I/O adapters implement user-orientedinput/output through, for example, software drivers and computerhardware for controlling output to display devices such as computerdisplay screens, as well as user input from user input devices (281)such as keyboards and mice. The example computer (252) of FIG. 2includes a video adapter (209), which is an example of an I/O adapterspecially designed for graphic output to a display device (280) such asa display screen or computer monitor. The video adapter (209) isconnected to processor (256) through a high speed video bus (264), busadapter (258), and the front side bus (262), which is also a high speedbus.

The exemplary computer (252) of FIG. 2 includes a communications adapter(267) for data communications with other computers (282) and for datacommunications with a data communications network (200). Such datacommunications may be carried out serially through RS-232 connections,through external buses such as a Universal Serial Bus (‘USB’), throughdata communications data communications networks such as IP datacommunications networks, and in other ways as will occur to those ofskill in the art. Communications adapters implement the hardware levelof data communications through which one computer sends datacommunications to another computer, directly or through a datacommunications network. Examples of communications adapters useful incomputers configured for determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present inventioninclude modems for wired dial-up communications, Ethernet (IEEE 802.3)adapters for wired data communications network communications, and802.11 adapters for wireless data communications network communications.

For further explanation, FIG. 3 sets forth a block diagram of anexemplary system for determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system (101) according to embodiments of the presentinvention. The system of FIG. 3 includes an event and alert analysismodule (124). The event and alert analysis module (124) of FIG. 3receives in an event queue (306) a plurality of events (302) from one ormore components of a distributed processing system (101). A component ofa distributed processing system according to embodiments of the presentinvention may be a device of the distributed processing system or aprocess running on a device of the distributed processing. Suchcomponents are often capable of some form of event transmission, oftenfor error or status reporting.

An event according to embodiments of the present invention is anotification of a particular occurrence in or on a component of thedistributed processing system. Such events are sent from the componentupon which the occurrence occurred or another reporting component to anevent and alert analysis module according to the present invention.Often events are notifications of errors occurring in a component of thedata processing system. Events are often implemented as messages eithersent through a data communications network or shared memory. Typicalevents for event and alert analysis according to embodiments of thepresent invention include attributes such as an occurred time, a loggedtime, an event type, an event ID, a reporting component, location, and asource component, and other attributes. An occurred time is the time atwhich the event occurred on the component. A logged time is the time theevent was included in the event queue (306) and is typically insertedinto the event by a monitor. An event type is an indication of the typeof the event such as for example, power error, link failure error,errors related to not receiving messages or dropping packets and so onas will occur to those of skill in the art. A location could be a node,mode, link, or other part of the system representing where the eventoccurred. An event ID is a unique identification of the event. Areporting component is an identification of the component that reportedthe event. A source component is an identification of the component uponwhich the event occurred. In many cases, but not all, the reportingcomponent and source component are the same component of the distributedprocessing system.

The event and analysis module (124) of FIG. 3 also includes a checkpointmanager (299) that is configured to process and manage checkpoints forthe incident analyzers.

In the example of FIG. 3, the monitor (204) receives events fromcomponents of the distributed processing system and puts the receivedevents (302) in the event queue (306). The monitor (204) of FIG. 3 mayreceive events from components of the distributed processing system ontheir motion, may periodically poll one or more of the components of thedistributed processing system, or receive events from components inother ways as will occur to those of skill in the art.

The system of FIG. 3 also includes an event analyzer (208). The eventanalyzer (208) of FIG. 3 is a module of automated computing machineryconfigured to identify alerts in dependence upon received events. Thatis, event analyzers typically receive events and produce alerts. In manyembodiments, multiple event analyzers are implemented in parallel. Oftenevent analyzers are assigned to a particular pool of events and may befocused on events from a particular component or caused by a particularoccurrence to produce a more concise set of alerts.

As mentioned above, in some embodiments of the present invention, morethan one event analyzer may operate in parallel. As such, each eventanalyzer may maintain one or more events pools for determining a numberof unique events in an events pool according to embodiments of thepresent invention. Assigning by the event analyzer the events to anevents pool may therefore include selecting only events from one or moreparticular components. In such embodiments, particular components may beselected for a particular events pool to provide events associated witha particular period of time from a particular set of one or morecomponents.

Assigning by the event analyzer the events to an events pool may also becarried out by selecting only events of a particular event type. In suchembodiments, particular events may be selected for a particular eventspool to provide events associated with a particular period of time froma particular set of event types. The event analyzer (208) in the exampleof FIG. 3 identifies in dependence upon the event analysis rules (310)and the events assigned to the events pool, one or more alerts (314).Event analysis rules (310) are a collection of predetermined rules formeaningfully parsing received events to identify relevant alerts independence upon the events.

The event analysis rules (310) of FIG. 3 include event arrival rules(330), events pool operation rules (332), event suppression rules (334),and events pool closure rules (336). The event arrival rules (330) areconfigurable predetermined rules for identifying alerts in dependenceupon events in real time as those events are assigned to the eventspool. That is, the event arrival rules (330) identify alerts independence upon events before closing the events pool. Such rules aretypically predetermined to identify particular alerts in dependence uponattributes of those events. Event arrival rules may for example dictateidentifying a particular predetermined alert for transmission to asystems administrator in dependence upon a particular event type orcomponent type for the event or other attribute of that event. Suchrules are flexible and may be tailored to a particular distributedcomputing system and its functions.

An alert according to embodiments of the present invention is a refinedidentification of an occurrence, such as an error based upon more thanone event, and therefore provides an identification of the occurrence inthe context of its operation in the distributed processing system. Oftenan alert may be a notification of a particular error type of occurrencethat is identified in dependence upon the plurality of events receivedfrom one or more components of the data processing system, such as, forexample, a link failure among a plurality of devices each of which areproducing many events based upon the single link failure, or a powerfailure provoking thousands of events, and so on.

Alerts are often implemented as messages to be sent through a datacommunications network or shared memory. Typical alerts according toembodiments of the present invention have attributes attached to thembased upon the attributes of the events received from which they areidentified.

The events pool operation rules (332) are configurable predeterminedrules for controlling the operations of the events pool. Such rulesincludes rules identifying the initial predetermined period of time foreach events pool, rules dictating the length of time extended to thepool upon the assignment of each new event to the pool, rules dictatingthe minimum time an event must be in a pool before that event isincluded in a collection of events when the pool is closed, rulesdictating the amount to extend the initial predetermined period of timebased on an arrival rate of events assigned to an events pool, rulesgoverning the closing of an events pool, and others as will occur tothose of skill in the art. Such rules are flexible and may be tailoredto a particular distributed computing system and its functions.

The event suppression rules (334) are configurable predetermined rulesfor suppressing one or more events in a closed pool of events used inidentifying alerts. That is, often events in the closed events pool maybe duplicate events, redundant events, or otherwise unnecessary orunhelpful events in identifying alerts. Such suppression rules aretypically predetermined to delete, drop, or otherwise ignore thosesuppressed events. Event suppression rules may for example dictate thatmore than a threshold number of events of a particular event type orcomponent type are to be suppressed. Such rules are also flexible andmay be tailored to a particular distributed computing system and itsfunctions.

The events pool closure rules (336) are configurable predetermined rulesfor identifying alerts in dependence upon unsuppressed events in theclosed events pool and alerts identified by the event arrival rules.That is, events pool closure rules identify new alerts in dependenceupon one or more or even all of the unsuppressed events in the closedevents pool. The events pool closure rules also identify alerts independence upon the alerts identified by the event arrival rules (330)or a combination of the alerts identified by the event arrival rules(330) and one or more of the unsuppressed events in the closed eventspool.

The event analyzer (208) in the example of FIG. 3 sends all the alerts(314) identified by the event analyzer (208) to an alert analyzer (218).The alert analyzer of FIG. 3 is a module of automated computingmachinery capable of identifying alerts for transmission from events andother alerts, identifying additional alerts for transmission, andsuppressing unnecessary, irrelevant, or otherwise unwanted or unhelpfulalerts identified by the event analyzer. That is, alert analyzerstypically receive alerts and events and produce or forward alerts independence upon those alerts and events. In many embodiments, alertanalyzers are implemented in parallel. The alerts (316) in the exampleof FIG. 3 are sent from the event analyzer (208) to the alert analyzer(218) through an alerts queue (316).

The alert analyzer (218) of FIG. 3 assigns each of the identified alerts(314) to an alerts pool (324). An alerts pool is a collection of alertsorganized by the time of one or more the events causing the alert to beidentified, the time the alert is identified, or other time as willoccur to those of skill in the art. That is, alerts pools are acollection of alerts organized by time. Such alerts pools often providethe ability to analyze groups alerts identified and included in thealerts pool according to some time. Often such alerts pools are usefulin identifying fewer and more relevant alerts in dependence uponmultiple related events and multiple related alerts.

The alert analyzer (218) of FIG. 3 determines in dependence upon alertanalysis rules (322) and the alerts in the alerts pool whether tosuppress any alerts. Suppressing an alert is typically carried out bydropping the alert, deleting the alert or otherwise ignoring or nottransmitting the suppressed alert to a component of the distributedprocessing system.

The alert analysis rules (322) are a collection of rules for suppressingone or more alerts to provide a more relevant set of alerts fortransmission to a component of the distributed processing system, suchas for example, for display to a systems administrator and to identifyadditional alerts for transmission to one or more components of thedistributed processing system. Alert analysis rules for example maydictate that duplicate alerts are to be suppressed, alerts of aparticular type for transmission to a particular component are to besuppressed, alerts of a particular type be transmitted to a particularcomponent are to be suppressed and so on as will occur to those of skillin the art. Such alerts may be more meaningful to a component of thedistributed processing system for automated error recovery or for asystems administrator who may otherwise be less informed by a number ofraw unanalyzed alerts.

The alert analyzer (218) of FIG. 3 also has access to the events queue(306). The alert analyzer (218) of FIG. 3 in dependence upon the alertanalysis rules may, in some embodiments select events from the eventsqueue and determine whether to suppress any alerts in dependence uponthe selected events. That is, alert analysis rules may also take intoaccount events and their attributes for suppressing alerts and foridentifying additional alerts for transmission to one or morecomponents. Such events may be related to the alerts in the alerts poolor independent from such alerts.

The alert analyzer (218) of FIG. 3 transmits the unsuppressed alerts toone or more components of the distributed processing system. The alertanalyzer may transmit the unsuppressed alerts to one or more componentsof the distributed processing system by sending the alert as a messageacross a data communications network, through shared memory, or in otherways as will occur to those of skill in the art. In the example of FIG.3, the unsuppressed alerts (320) are transmitted to the terminal (122)for display to the systems administrator (128).

The alert analyzer (218) of FIG. 3 is also configured to identify independence upon alert analysis rules (322), the alerts in the alertspool (324), and selected events (306) one or more additional alerts andtransmitting the additional alerts to one or more components of thedistributed processing system. The additional alerts may include one ormore alerts not identified by the event analyzer. Such additional alertsmay provide additional information to a component of the distributedprocessing system of a systems administrator.

In the system of FIG. 3, events (302) are received and analyzed by eventanalyzers (208) with event analysis rules (310). Based on the eventanalysis rules (310), the event analyzers (208) generate the alerts(314). These alerts may be sent to a delivery queue (399) for immediatedelivery to the system administrator (128) and the distributedprocessing system (101). These alerts may also be sent to alertanalyzers (218) for further processing and generation of additionalalerts (320), which may also be provided to the delivery queue (399).The event and alert analysis module (124) also includes an alertdatabase (397) for recording alerts that have generated by the event andalert analysis module (124).

In the example of FIG. 3, the event analyzer (208) is also configured todetermine a number of unique events in a plurality of events for eventprocessing. Specifically, the event analyzer (208) is configured toidentify within a plurality of events, attribute combination entries oflocation identifications and event types. Each attribute combinationentry has one location identification and a set of unique event typescorresponding to the location identification. The event analyzer (208)is also configured to analyze each location identification in eachattribute combination entry according to a sequence of the attributecombination entries. For each location identification of each attributecombination entry, the event analyzer (208) is configured to determinewhether the set of unique event types within the attribute combinationentry includes an event type not selected for pairing with anotherlocation identification in an attribute pair. If the set of unique eventtypes includes an event type not selected for pairing with anotherlocation identification in an attribute pair, the event analyzer (208)is configured to select the event type for pairing with the locationidentification in an attribute pair and to create an attribute pair ofthe selected event type and the location identification. The eventanalyzer is also configured to count the attribute pairs, where thenumber of attribute pairs is the number of unique events in theplurality of events.

In the example of FIG. 3, the alert analyzer (218) is also configured todetermine a number of unique alerts in a plurality of alerts for alertprocessing. Specifically, the alert analyzer (218) is configured toidentify within a plurality of alerts, attribute combination entries oflocation identifications and alert types. Each attribute combinationentry has one location identification and a set of unique alert typescorresponding to the location identification. The alert analyzer (218)is also configured to analyze each location identification in eachattribute combination entry according to a sequence of the attributecombination entries. For each location identification of each attributecombination entry, the alert analyzer (218) is configured to determinewhether the set of unique alert types within the attribute combinationentry includes an alert type not selected for pairing with anotherlocation identification in an attribute pair. If the set of unique alerttypes includes an alert type not selected for pairing with anotherlocation identification in an attribute pair, the alert analyzer (218)is configured to select the alert type for pairing with the locationidentification in an attribute pair and to create an attribute pair ofthe selected alert type and the location identification. The alertanalyzer is also configured to count the attribute pairs, where thenumber of attribute pairs is the number of unique alerts in theplurality of alerts. Readers of skill in the art, will also realize thatother attributes, such as attributes of an alert, may be utilized by thealert analyzer to determine the number of unique alerts in a pluralityof alerts.

As mentioned above, determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention mayinclude assigning events to an events pool and those pools areadministered according to embodiments of the present invention. Forfurther explanation, FIG. 4 sets forth a diagram illustrating assigningevents to an events pool according to embodiments of the presentinvention. An events pool (312) is a collection of events organized bythe time of either their occurrence, by the time they are logged in theevent queue, included in the events pool, or other time as will occur tothose of skill in the art. That is, events pools are a collection ofevents organized by time. Such events pools often provide the ability toanalyze a group of time related events and to identify alerts independence upon them. Often such events pools are useful in identifyingfewer and more relevant alerts in dependence upon multiple relatedevents.

Events pools according to embodiments of the present invention aretypically operated according to events pool operation rules which arethemselves often included in event analysis rules. Such events pooloperation rules are configurable predetermined rules for controlling theoperations of the events pool. Such rules includes rules identifying theinitial predetermined period of time for each events pool, rulesdictating the length of time extended to the pool upon the assignment ofeach new event to the pool, rules dictating the minimum time an eventmust be in a pool before that event is included in a collection ofevents when the pool is closed, rules dictating the amount to extend theinitial predetermined period of time based on an arrival rate of eventsassigned to an events pool, rules governing the closing of an eventspool, and others as will occur to those of skill in the art. Such rulesare flexible and may be tailored to a particular distributed computingsystem and its functions.

Events are often assigned to an events pool according to their loggedtime. That is, events are typically inserted into the events pool in theorder that they are received in the event queue. In the example of FIG.4, the timing of the events pool (312) is initiated when the first event‘Event 0’ (400) is assigned to the events pool (312) at time t₀. Theevents pool of FIG. 4 is initiated for a predetermined initial period oftime from t₁ to t_(f). That is, upon receiving the first event ‘Event 0’(400) the events pool of FIG. 4 has a predetermined initial period oftime beginning at t₁ and ending at t_(f). The predetermined initialperiod of time may be configured in dependence upon a number of factorsas will occur to those of skill in the art such as, the number ofcomponents in the distributed processing system, the frequency ofreceiving events, the types of events typically received and so on aswill occur to those of skill in the art.

In the example FIG. 4, the initial period of time is extended for eachnew event assigned to the events pool during the predetermined initialperiod from t₁ to t_(f) by a particular period of time assigned to theevent. In the example of FIG. 4 upon assigning ‘Event 1’ (402) to theevents pool (312) the predetermined initial period of time t₀-t_(f) isextended by ‘Extension 1’ (406) having a time of e1 thereby creating anew time for closing the events pool (312) at t_(f+e1) if no otherevents are assigned to the pool before t_(f+e1). Similarly, in theexample of FIG. 4 upon assigning ‘Event 2’ (404) to the events poolhaving a time of e2, the now extended period of time from t₀-t_(f+e1) isextended again by extension 2 (406) thereby establishing a new time forclosing the pool at time t_(f+e1+e2) if no other events are assigned tothe pool before t_(f+e1+e2) or before some maximum time for the eventspool has expired. In this manner, the events pool is extended with eachreceived event until a collection of events that may be usefully used toidentify alerts is assigned to the events pool. According to embodimentsof the present invention, the predetermined initial period of time maybe extended based on an arrival rate at which events are assigned to anevents pool.

In typical embodiments of the present invention, events pools may have amaximum duration that can no longer be extended. In such cases, arequirement may exist that an event that has not resided in the eventspool for a threshold period of time be moved to a next events pool. Insome embodiments, the attributes of such an event that is moved to thenext events pool are used for relevant alert delivery with the initialevents pool and in other embodiments; the attributes of such an eventare used for relevant alert delivery with the next events pool to whichthat event is moved.

In the example of FIG. 4, when conditions are met to close the pool anevents analyzer determines for each event (400, 402, 404) in the eventspool (312) whether the event has been in the pool for its predeterminedminimum time for inclusion in a pool. If the event has been in the poolfor its predetermined minimum time, the event is included in the closedpool for event analysis for relevant alert delivery. If the event hasnot been in the pool for its predetermined minimum time, the event isevicted from the closed pool and included a next pool for event analysisfor relevant alert delivery.

In many embodiments, a plurality of events pools may be used in paralleland one or more of such events pools are assigned to a particular eventsanalyzer. In such embodiments, events analyzers may be directed toevents in events pools having particular attributes.

As mentioned above, determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention mayinclude assigning alerts to an alerts pool and those pools areadministered according to embodiments of the present invention.

For further explanation, FIG. 5 sets forth a diagram illustratingassigning alerts to an alerts pool according to embodiments of thepresent invention. The alerts pool (324) of FIG. 5 operates in a mannersimilar to the events pool of FIG. 4. That is, the alerts pool accordingto the example of FIG. 5 includes alerts and the timing of the alertspool begins with the first alert ‘Alert 0’ (500) at time t₀ and isconfigured to have a predetermined initial period of time t₀-tf. In theexample of FIG. 5, the initial period of time is extended for each newalert assigned to the alerts pool in the predetermined initial periodfrom t₁ to t_(f) by a particular period of time assigned to the alert.In the example of FIG. 5, upon assigning ‘Alert 1’ (502) to the alertspool (324) the predetermined initial period of time t₀-t_(f) is extendedby ‘Extension 1’ (506) having a time of e1 thereby creating a new timefor closing the alerts pool (324) at t_(f+e1) if no other alerts areassigned to the pool before t_(f+e1). Similarly, in the example of FIG.5 upon assigning ‘Alert 2’ (504) to the alerts pool having a time of e2,the now extended period of time from t₀-t_(f+e1) is extended again by‘Extension 2’ (506) thereby establishing a new time for closing the poolat time t_(f+e1+e2) if no other alerts are assigned to the pool beforet_(f+e1+e2) or before some maximum time for the alerts pool has expired.According to embodiments of the present invention, the predeterminedinitial period of time may be extended based on an arrival rate at whichalerts are assigned to an alerts pool.

In typical embodiments of the present invention, alerts pools may have amaximum duration that can no longer be extended. In such cases, arequirement may exist that an alert that has not resided in the alertspool for a threshold period of time be moved to a next alerts pool. Insome embodiments, the attributes of such an alert that is moved to thenext alerts pool are used for relevant alert delivery according toembodiments of the present invention with the initial alerts pool and inother embodiments, the attributes of such an alert are used for relevantalert delivery with the next alerts pool to which that alert is moved.

In the example of FIG. 5, when conditions are met to close the pool analerts analyzer determines for each alert (500, 502, 504) in the pool(324) whether the alert has been in the pool for its predeterminedminimum time for inclusion in a pool. If the alert has been in the poolfor its predetermined minimum time, the alert is included in the closedpool for alert analysis for relevant alert delivery according toembodiments of the present invention. If the alert has not been in thepool for its predetermined minimum time, the alert is evicted from theclosed pool and included a next pool for alert analysis for relevantalert delivery according to embodiments of the present invention.

In many embodiments, a plurality of alerts pools may be used in paralleland one or more of such alerts pools are assigned to a particular alertsanalyzer. In such embodiments, alerts analyzers may be directed toalerts in alerts pools having particular attributes.

As mentioned above, determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention mayinclude the administration of one or more pools of incidents such asevents, alerts or other incidents as will occur to those of skill in theart. For further explanation, FIG. 6 sets forth a flow chartillustrating an example method of administering incident pools forincident processing in a distributed processing system according toembodiments of the present invention. The method of FIG. 6 includesreceiving (602) in an event queue a plurality of events (302) from oneor more components of a distributed processing system. Attributes ofevents useful in determining a number of unique events in an events poolfor incident analysis in a distributed processing system according toembodiments of the present invention may include an occurred time, alogged time, an event type, an event ID, a reporting component, alocation, and a source component.

Receiving (602) in an event queue a plurality of events (302) from oneor more components of a distributed processing system may be carried outby receiving an event initiated by one or more components of the dataprocessing system and storing the event in the event queue according tothe time in which the event occurred or according to the time the eventwas received. Receiving (602) in an event queue a plurality of events(302) from one or more components of a distributed processing systemalso may be carried out by polling a component for status and receivingin response an event and storing the event in the event queue accordingto the time in which the event occurred or according to the time theevent was received.

The method of FIG. 6 also includes assigning (604) by an event analyzereach received event to an events pool (312). In some embodiments of thepresent invention, assigning (604) by an event analyzer each receivedevent (302) to an events pool (312) may be carried out by assigningevents to the events pool according to the logged time. Assigning (604)by an event analyzer each received event (302) to an events pool (312)may also be carried out in dependence upon attributes of the event. Suchattributes may include an identification or type of the component uponwhich an occurrence occurred to create the event, the reportingcomponent of the event, the event ID, the event type, and so on as willoccur to those of skill in the art.

An events pool according to the method of FIG. 6 includes eventsoccurring during a predetermined initial period of time and in theexample of FIG. 6 assigning (604) by the event analyzer each receivedevent to an events pool includes extending (626) for each event assignedto the events pool the predetermined initial period of time by aparticular period of time assigned to the event.

The event analyzer includes event analysis rules (310) including, eventarrival rules, events pool operation rules, event suppression rules, andevents pool closure rules. Event arrival rules are configurablepredetermined rules for identifying alerts in dependence upon events inreal time as those events are assigned to the events pool. That is,event arrival rules identify alerts in dependence upon events beforeclosing the events pool. Such rules are flexible and may be tailored toa particular distributed computing system and its functions.

An alert according to embodiments of the present invention is a refinedidentification of an occurrence, such as an error based upon more thanone event, and therefore provides an identification of the occurrence inthe context of its operation in the distributed processing system. Oftenan alert may be a notification of a particular error type of occurrencethat is identified in dependence upon the plurality of events receivedfrom one or more components of the data processing system, such as, forexample, a link failure among a plurality of devices each of which areproducing many events based upon the single link failure, or a powerfailure provoking thousands of events, and so on.

Alerts are often implemented as messages to be sent through a datacommunications network or shared memory. Typical alerts according toembodiments of the present invention have attributes attached to thembased upon the attributes of the events received from which they areidentified.

Events pool operation rules are configurable predetermined rules forcontrolling the operations of the events pool. Such rules includes rulesidentifying the initial predetermined period of time for each eventspool, rules dictating the length of time extended to the pool upon theassignment of each new event to the pool, rules dictating the minimumtime an event must be in a pool before that event is included in acollection of events when the pool is closed, rules governing theclosing of an events pool, and others as will occur to those of skill inthe art. Such rules are flexible and may be tailored to a particulardistributed computing system and its functions.

Event suppression rules are configurable predetermined rules forsuppressing one or more events in a closed pool of events used inidentifying alerts. That is, often events in the closed events pool maybe duplicate events, redundant events, or otherwise unnecessary orunhelpful events in identifying alerts. Such suppression rules aretypically predetermined to delete, drop, or otherwise ignore thosesuppressed events. Event suppression rules may for example dictate thatmore than a threshold number of events of a particular event type orcomponent type are to be suppressed. Such rules are also flexible andmay be tailored to a particular distributed computing system and itsfunctions.

Events pool closure rules are configurable predetermined rules foridentifying alerts in dependence upon unsuppressed events in the closedevents pool and alerts identified by the event arrival rules. That is,events pool closure rules identify new alerts in dependence upon one ormore or even all of the unsuppressed events in the closed events pool.The events pool closure rules also identify alerts in dependence uponthe alerts identified by the event arrival rules or a combination of thealerts identified by the event arrival rules and one or more of theunsuppressed events in the closed events pool.

The method of FIG. 6 also includes identifying (610) by the eventanalyzer in dependence upon the event arrival rules and the eventsassigned to the events pool one or more alerts (314). Identifying (610)by the event analyzer in dependence upon the event arrival rules and theevents assigned to the events pool one or more alerts (314) may becarried out by identifying alerts in dependence upon one or moreattributes of the events as that event is assigned to the events pool.Identifying (610) by the event analyzer in dependence upon the eventarrival rules and the events assigned to the events pool one or morealerts (314) may be carried by comparing the attributes of the events tothe event arrival rules and identifying as a result of the comparisonone or more alerts. Such attributes may include the type of componentfrom which the event was received, the type of component creating theevent, the identification of the component creating the event, the timethe event was created or received, an error reported in the event, andmany others as will occur to those of skill in the art.

The method of FIG. 6 also includes closing (612), by the event analyzerin dependence upon the events pool operation rules, the events pool(312). Closing (612), by the event analyzer in dependence upon theevents pool operation rules, the events pool (312) may be carried out bydetermining that conditions dictated by the events pool operation ruleshave been met to stop assigning new events to the events pool andidentifying in dependence upon those events pool operation rules theparticular events that are included in the closed pool of events.

Closing the events pool may be carried out by determining that theinitial period of time for the events pool and any particular periods oftime for events received in the events pool extended to the initialperiod of time have expired. In such cases, if no new events arereceived prior to the expiration of the initial period of time for theevents pool and any particular periods of time for events received inthe events pool extended to the initial period of time the pool isclosed.

Closing the events pool may also be carried out by determining that amaximum duration for the events pool has expired. In such cases,regardless of the number of new events being received after a maximumduration for the events pool has expired, the pool is closed. In suchembodiments, a maximum duration for the events pool prevents the eventspool from including more events than are useful for relevant alertdelivery according to embodiments of the present invention.

The method of FIG. 6 also includes determining (614), by the eventsanalyzer in dependence upon the event suppression rules, whether tosuppress one or more events in the closed events pool (312). Determining(614), by the events analyzer in dependence upon the event suppressionrules, whether to suppress one or more events in the closed events pool(312) may be carried out by determining in dependence upon theattributes of one or more events in the closed pool whether to delete,drop, or otherwise ignore one or more of the events in the closed pool.

The method of FIG. 6 includes identifying (616) by the event analyzer independence upon the events pool closure rules and any unsuppressedevents assigned to the events pool, one or more additional alerts (617).Identifying (616) by the event analyzer in dependence upon the eventspool closure rules and any unsuppressed events assigned to the eventspool, one or more additional alerts (617) may be carried out byidentifying alerts in dependence upon one or more attributes of theevents as that event is assigned to the events pool. Identifying (616)by the event analyzer in dependence upon the events pool closure rulesand any unsuppressed events assigned to the events pool, one or moreadditional alerts (617) may be carried out by selecting the unsuppressedevents for the events pool, comparing the attributes of the unsuppressedevents of the events pool to the pool closure rules, and identifying asa result of the comparison one or more additional alerts. Suchattributes may include the type of component from which one or more ofthe unsuppressed events are received, the type of components creatingthe unsuppressed events, the identification of the component creatingthe unsuppressed events, the time the events were created or received,one or more errors reported by the events event, the number of events inthe pool, and many others as will occur to those of skill in the art.

The method of FIG. 6 includes sending (618) by the event analyzer to analert analyzer all the alerts identified by the event analyzer. Sending(618) by the event analyzer to an alert analyzer all the alerts (314)identified by the event analyzer may be carried out by sending a messagecontaining the alerts from the event analyzer to the alert analyzer.Such a message may be sent from the event analyzer to the alert analyzeracross a network, through shared memory, or in other ways as will occurto those of skill in the art.

The method of FIG. 6 includes assigning (620) by the alert analyzer theidentified alerts to an alerts pool (324). An alerts pool according tothe method of FIG. 6 has a predetermined initial period of time and inthe example of FIG. 6 assigning (620) by the alert analyzer theidentified alerts to an alerts pool (324) includes extending for eachalert assigned to the alerts pool the predetermined initial period oftime by a particular period of time assigned to the alert. Assigning(620) by the alert analyzer the identified alerts to an alerts pool(324) also may be carried out in dependence upon attributes of thealerts. Such attributes may include an identification or type of thecomponent upon which an occurrence occurred to create the event that wasused to identify the alert, the alert ID, the alert type, and so on aswill occur to those of skill in the art.

The method of FIG. 6 includes determining (622) by the alert analyzer independence upon alert analysis rules (322) and the alerts in the alertspool whether to suppress any alerts. Determining (622) by the alertanalyzer in dependence upon alert analysis rules (322) and the alerts inthe alerts pool whether to suppress any alerts may be carried out independence upon one or more attributes of the alerts. Such attributesmay include an identification or type of the component upon which anoccurrence occurred to create the event that was used to identify thealert, the alert ID, the alert type, and so on as will occur to those ofskill in the art. In such embodiments, determining (622) by the alertanalyzer in dependence upon alert analysis rules (322) and the alerts inthe alerts pool whether to suppress any alerts may be carried out bycomparing the attributes of the alerts in the alerts pool to the alertanalysis rules and identifying as a result of the comparison one or morealerts for suppression according to the event analysis rules.

The method of FIG. 6 includes delivering (628) the unsuppressed alertsto one or more components of the distributed processing system.Delivering (628) the unsuppressed alerts to one or more components ofthe distributed processing system may be carried out by sending amessage containing the alert to one or more components of thedistributed processing system. In many cases, an alert may be sent as amessage to a systems administrator advising the systems administrator ofone or more occurrences within the distributed processing system.

As mentioned above, alert analysis rules may select additional alerts orsuppress alerts in dependence upon events. In such embodiments,determining whether to suppress any alerts includes selecting events anddetermining whether to suppress any alerts in dependence upon theselected events. The method of FIG. 6 therefore also includesidentifying (626) by the alert analyzer in dependence upon alertanalysis rules (322), the alerts in the alerts pool (324), and anyselected events one or more additional alerts and in the method of FIG.6, delivering (628) the unsuppressed alerts also includes delivering(630) any additional alerts to one or more components of the distributedprocessing system.

For further explanation, FIG. 7 sets forth a flow chart illustrating anexemplary method of determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention.

An incident is a generic term used in this specification to mean anidentification or notification of a particular occurrence on a componentof a distributed processing system such as events, a refinedidentification of an occurrence often based on events such as an alert,or other notifications as will occur to those of skill in the art.

In the example of FIG. 7, an incidents pool (785) includes a plurality(786) of ‘N’ number of incidents. For ease of explanation, only a firstincident (730), a second incident (732), and an Nth incident (733) areillustrated. Each incident has a corresponding location identificationand an incident type. A location identification may be any identifyinglocation associated with the incident, such as a designation of where anincident represented by the incident originated from within thedistributed computing system (e.g., a node, a module, or a link). Anincident type is an indication of the type of the incident such as forexample, power error, link failure error, errors related to notreceiving messages or dropping packets and so on as will occur to thoseof skill in the art. For example, the first incident (730) has alocation identification (735) of ‘L1’ and an incident type (736) of‘I2’.

The method of FIG. 7 includes an incident analyzer (700) identifying(702) within the plurality of incidents (786), attribute combinationentries (760) of location identifications and incident types. Eachattribute combination entry (760) has at least one locationidentification (778) and a set (779) of unique incident typescorresponding to the at least one location identification (778). Thatis, the attribute combination entries are keyed to the locationidentification and have values that are the set of unique incident typesassociated with that location identification. For example, in FIG. 7,the attribute combination entries (760) include a first pair (790)having a location identification of ‘L1’ and a set of incident typesthat include ‘I1’ and ‘I2’. The attribute combination entries (760) alsoinclude a second pair (791) having a location identification of ‘L5’ and‘L6’ and a set of incident types that include ‘I3’ and ‘I4’. In thesecond pair (791), the ‘L5’ location identification and the ‘L6’location identification had an identical set of incident types and weretherefore merged together in the attribute combination entry to compactand reduce the size of the attribute combination entries (760). Thethird pair (792) includes the ‘L2’ location identification and ‘L3’location identification, both having the identical set of incident typesthat includes ‘I2’, ‘I3’, and ‘I7’. The fourth pair (793) includes the‘L4’ location identification and the ‘L7’ location identification andthe identical set of incident types that includes ‘I1’, ‘I3’, ‘I7’, and‘I8’. As will be explained in FIG. 10, in other embodiments of theinvention, the location identifications having identical sets ofincident types may initially not be merged or may not be merged all at,in which case the location identifications are placed in separateattribute combination entries.

The method of FIG. 7 also includes the incident analyzer (700) analyzing(704) each location identification (778) in each attribute combinationentry (760) according to a sequence (780) of the attribute combinationentries. A sequence is the order that the location identifications orthe attribute combination entries are analyzed. In the example of FIG.7, the sequence (780) start with the location identifications in thefirst pair (790), and proceeds to the second pair (791), to the thirdpair (792), and ends with the fourth pair (793). As will be explained ingreater detail in FIG. 8, the particular sequence that the locationidentifications and attribute combination entries are processed mayimpact the analysis of the attribute combination entries and ultimatelythe number of attribute pairs and the determination of the number ofunique incidents in the plurality of incidents.

Analyzing (704) each location identification (778) in each attributecombination entry (760) according to the sequence (780) of the attributecombination entries includes for each location identification of eachattribute combination entry: determining (706) whether the set of uniqueincident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair (770). Determining (706) whether theset of unique incident types within the attribute combination entryincludes an incident type not selected for pairing with another locationidentification in an attribute pair (770) may be carried out by storingwithin a data structure, an incident type that is selected for pairingwith another location identification in an attribute pair; and examiningthe data structure that contains the previously selected incident typesto determine if a currently selected incident type is one of thepreviously selected incident types.

Analyzing (704) each location identification (778) in each attributecombination entry (760) according to a sequence (780) of the attributecombination entries also includes for each location identification ofeach attribute combination entry: if the set of unique incident typesincludes an incident type not selected for pairing with another locationidentification in an attribute pair, selecting (710) the incident typefor pairing with the location identification in an attribute pair andcreating (712) an attribute pair of the selected incident type and thelocation identification. Selecting (710) the incident type for pairingwith the location identification in an attribute pair may be carried outby randomly picking one of the incident types; or following a specificpattern for picking one of the incident types from the entry, such apicking the first incident type or the lowest number incident type.

Readers of skill in the art will realize that any number of specificpatterns may be utilized for picking one of the incident types from theentry. For example, in a particular embodiment, the incident analyzer(700) is configured to track the number of times that a particularincident type appears in sets of incidents of the attribute combinationentries. The incident analyzer may also be configured to create a listthat lists each incident type in increasing order of popularity. Thatis, the list specifies an increasing order of popularity that eachincident type appears in the combination entries. The incident analyzer(700) may use this list in selecting (710) the incident type for pairingwith the location identification in an attribute pair by picking,according to the list, the least popular incident type that is stillavailable for selection. By picking the least most popular incidenttypes first, the incident analyzer may increase the likelihood that moreincident types will be available for selection for pairing with thelater location identifications.

Creating (712) an attribute pair of the selected incident type and thelocation identification may be carried out by storing both the locationidentification and the selected incident type in a unique incidentslist; and storing an indication that the location identification and theselected incident type correspond to each other within the uniqueincidents list.

The method of FIG. 7 also includes the incident analyzer (700) counting(714) the attribute pairs (770). Counting (714) the attribute pairs(770) may be carried out by maintaining a counter that increments witheach addition of an attribute pair in a unique incidents list; orperiodically counting number of location identifications, incidenttypes, or attribute pairs within a unique incidents list. In the exampleof FIG. 7, the number (799) of attribute pairs is the number of uniqueincidents in the plurality of incidents (786).

For example, each incident in the plurality of incidents may each haveone location identification of the group: ‘L1’, ‘L2’, ‘L3’, ‘L4’, ‘L5’,‘L6’, and ‘L7’ and one incident type of the group: ‘I1’, ‘I2’, ‘I3’,‘I4’, ‘I7’, and ‘I8’. In this plurality of incidents, there may bemultiple incidents with the same combination of location identificationand incident type and there may be only one incident with the samecombination of location identification and incident type. Accordingly toembodiments of the present invention, the incident analyzer isconfigured to take this plurality of incidents and determine the numberof unique incidents (i.e., a unique incident type and a unique locationidentification). Continuing with this example, the incident analyzercreates the attribute combination entries (760) from this plurality ofincidents. The attribute combination entries (760) identify the uniqueincident types corresponding to a location identification. The incidentanalyzer (700) analyzes the attribute combination entries (760)according to a sequence.

In this example, if the incident analyzer (700) follows the sequence(780), the ‘L1’ location identification is analyzed first and anincident type is selected from the set of incident types in the firstattribute combination entry (790). Either ‘I1’ or ‘I2’ may be selectedbut in this example, ‘I1’ is selected for pairing with the ‘L1’ locationidentification in a first attribute pair (781). The incident analyzer(700) next analyzes the ‘L5’ location identification and selects ‘I3’incident type for a second attribute pair (782) then selects ‘I4’ forpairing with the ‘L6’ location identification in a third attribute pair(783). When the incident analyzer (700) analyzes the third attributecombination entry (792), the incident analyzer would determine that the‘I3’ incident type has already been selected for pairing with anotherlocation identification (i.e., ‘L5’). The ‘I2’ and ‘I7’ incident typesare both available for selection and pairing with the ‘L2’ and ‘L3’location identifications in a fourth attribute pair (784) and a fifthattribute pair (720), respectively. When the incident analyzer (700)analyzes the fourth attribute combination entry (793), the incidentanalyzer would determine that the ‘I1’, ‘I3’, and ‘I7’ are allunavailable for selection so the incident analyzer would select ‘I8’incident type for pairing with the ‘L4’ location identification in asixth attribute pair (721). As there are no more unselected incidenttypes in the fourth attribute combination entry (793), the incidentanalyzer is unable to select an incident type for pairing with the ‘L7’location identification. As a consequence, the ‘L7’ locationidentification is not included in the attribute pairs (770). Todetermine the number of unique incidents, the incident analyzer countsthe number of attribute pairs (770), which in this example is six. Thatis, the incident analyzer in this example may determine that there aresix unique incidents in the plurality of incidents.

Determining the number of unique incidents in a plurality of incidentsmay be useful for a variety of reasons. For example, an incidentprocessing system or an incident analyzer may have a ruleset that isdependent upon the number of unique incidents. A ruleset may contain aset of rules that define how to process incidents and generateappropriate actions based on the incidents. For example, a particularcondition of a ruleset may require that there be a set of N incidents inthe incidents pool where each incident has a unique incident type and aunique location identification. This condition may be useful indetermining a ‘breadth of failure’ of the system. In a particularembodiment, the incident analyzer (700) of FIG. 7 is configured todetermine if this condition is satisfied.

For further explanation, FIG. 8 sets forth a flow chart illustrating anadditional method of determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention. Themethod of FIG. 8 is similar to the method of FIG. 7 in that the methodof FIG. 8 also includes: identifying (702) within the plurality ofincidents (786), attribute combination entries (760) of locationidentifications and incident types; analyzing (704) each locationidentification (778) in each attribute combination entry (760) accordingto a sequence (780) of the attribute combination entries; determining(706) whether the set of unique incident types within the attributecombination entry includes an incident type not selected for pairingwith another location identification in an attribute pair (770); if theset of unique incident types includes an incident type not selected forpairing with another location identification in an attribute pair,selecting (710) the incident type for pairing with the locationidentification in an attribute pair and creating (712) an attribute pairof the selected incident type and the location identification; andcounting (714) the attribute pairs (770).

The method of FIG. 8, however, also includes creating (802) the sequence(780) of the attribute combination entries (760). Creating (802) thesequence (780) of the attribute combination entries (760) includessorting (804) the attribute combination entries (760) based on one of anumber of incident types within each set of unique incident types and anumber of location identifications. Sorting (804) the attributecombination entries (760) based on one of a number of incident typeswithin each set of unique incident types and a number of locationidentifications may be carried out by sorting the sets of incident typeswith the fewest location identification pairings to be selected first.For example, the incident analyzer may determine which is smaller, thenumber of incident types or the number of location identifications andthen sort the attribute combination entries based on that determination.Sorting the sets of incident types with the fewest locationidentification pairs to be selected first may allow later locationidentifications to have more incident types to choose from.

For example, an incident analyzer processing a first attributecombination entry having a ‘L1’ location identification with a set ofincident types that only includes a ‘I1’ incident type and a secondattribute combination entry having a ‘L2’ location identification with aset of incident types that include ‘I1’, ‘I3’, ‘I4’, and ‘I6’. If thesecond pair is analyzed first and ‘I1’ is selected for pairing with the‘L2’ location identification then when the first attribute combinationentry is analyzed, the ‘I1’ incident type would not be available forpairing with the ‘L1’ location identification, resulting in only oneattribute pair of (‘L2’, ‘I1’). However, if the first pair is analyzedfirst following by the second attribute combination entry, two attributepairs may be created (‘L1’, ‘I1’), (‘L2’, ‘I3’). That is, the sequencethat the attribute combination entries are analyzed may impact thenumber of attribute pairs and ultimately the determined number of uniqueincidents in the plurality of incidents.

For further explanation, FIG. 9 sets forth a flow chart illustrating anadditional method of determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention. Themethod of FIG. 9 is similar to the method of FIG. 7 in that the methodof FIG. 9 also includes identifying (702) within the plurality ofincidents (786), attribute combination entries (760) of locationidentifications and incident types; analyzing (704) each locationidentification (778) in each attribute combination entry (760) accordingto a sequence (780) of the attribute combination entries; determining(706) whether the set of unique incident types within the attributecombination entry includes an incident type not selected for pairingwith another location identification in an attribute pair (770); if theset of unique incident types includes an incident type not selected forpairing with another location identification in an attribute pair,selecting (710) the incident type for pairing with the locationidentification in an attribute pair and creating (712) an attribute pairof the selected incident type and the location identification; andcounting (714) the attribute pairs (770).

In the method of FIG. 9, however, determining (706) whether the set ofunique incident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair (770) includes maintaining (902) alist (950) of selected incident types that have already been selectedfor pairing with a location identification in an attribute pair.Maintaining (902) a list (950) of selected incident types that havealready been selected for pairing with a location identification in anattribute pair may be carried out by storing within a data structure, anincident type that is selected for pairing with another locationidentification in an attribute pair.

In the method of FIG. 9, however, determining (706) whether the set ofunique incident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair (770) includes comparing (904) theset of unique incident types to the list of selected incident types.Comparing (904) the set of unique incident types to the list of selectedincident types may be carried out by examining the data structure thatcontains the previously selected incident types to determine if acurrently selected incident type is one of the previously selectedincident types.

For further explanation, FIG. 10 sets forth a flow chart illustrating anadditional method of determining a number of unique incidents in aplurality of incidents for incident processing in a distributedprocessing system according to embodiments of the present invention. Themethod of FIG. 10 is similar to the method of FIG. 7 in that the methodof FIG. 10 also includes identifying (702) within the plurality ofincidents (786), attribute combination entries (760) of locationidentifications and incident types; analyzing (704) each locationidentification (778) in each attribute combination entry (760) accordingto a sequence (780) of the attribute combination entries; determining(706) whether the set of unique incident types within the attributecombination entry includes an incident type not selected for pairingwith another location identification in an attribute pair (770); if theset of unique incident types includes an incident type not selected forpairing with another location identification in an attribute pair,selecting (710) the incident type for pairing with the locationidentification in an attribute pair and creating (712) an attribute pairof the selected incident type and the location identification; andcounting (714) the attribute pairs (770).

The method of FIG. 10 also includes the incident analyzer (700)identifying (1002) attribute combination entries (1040) having identicalsets of incident types. Identifying (1002) attribute combination entries(1040) having identical sets of incident types may be carried out byexamining the sets of incident types for a matching combination.

The method of FIG. 10 also includes the incident analyzer (700) merging(1004) the identified attribute combination entries (1040). Merging(1004) the identified attribute combination entries (1040) may becarried out by generating an attribute combination entry that includesall of the location identifications and the set of incident types sharedby all of the location identifications. In a particular embodiment, eachattribute combination entry has only one location identification. Thatis, location identifications having identical sets of incident types mayinitially not be merged or may not be merged at all, in which case thelocation identifications are each placed in a separate attributecombination entry. Said another way, when two attribute combinationentries are merged then the location identifications from those entriesare added together to form a set of unique location identifications andthe set of unique incident types that is common to the entries is usedfor the new combined entry.

The method of FIG. 10 also includes the incident analyzer (700)determining (1006) that the number (799) of unique incidents is greaterthan or equal to a predetermined number (1050). Determining (1006) thatthe number (799) of unique incidents is greater than or equal to apredetermined number (1050) may be carried out by comparing the number(799) to the predetermined number. For example, a particular ruleset mayhave a condition which requires that there be a set of N incidents inthe incidents pool where each incident has a unique incident type and aunique location identification. If the incident analyzer determines thatthere are N unique incidents, the incident analyzer may determine thatthe condition is satisfied.

The method of FIG. 10 also includes the incident analyzer (700)executing (1008) an action in response to determining that the number(799) of unique incidents is greater than or equal to the predeterminednumber (1050). Executing (1008) an action in response to determiningthat the number (799) of unique incidents is greater than or equal tothe predetermined number (1050) may be carried out by deleting,dropping, suppressing, or moving an incident, from the system, toanother component of the system, from an incidents pool, or to anincidents pool.

Executing (1008) an action in response to determining that the number(799) of unique incidents is greater than or equal to the predeterminednumber (1050) includes at least one of creating (1012) an alert andsuppressing (1014) an incident. Creating (1012) an alert may be carriedout by generating an alert if the number of attribute pairs is above aparticular predetermined number. An alert is a refinement of anoccurrence, such as an error, based upon more than one incident andtherefore provides an identification of the occurrence in the context ofits operation in the distributed processing system. Often an alert maybe a notification of a particular error type of occurrence that isidentified in dependence upon the plurality of incidents received fromone or more components of the data processing system, such as, forexample, a link failure among a plurality of devices each of which areproducing many incidents based upon the single link failure, or a powerfailure provoking thousands of incidents, and so on. Alerts are oftenimplemented as messages to be sent through a data communications networkor shared memory. Typical alerts according to embodiments of the presentinvention have attributes attached to them based upon the attributes ofthe incidents received from which they are identified.

Suppressing (1014) an incident may be carried out by based on incidentsuppression rules. Incident suppression rules are configurablepredetermined rules for suppressing one or more incidents. Suchsuppression rules are typically predetermined to delete, drop, orotherwise ignore those suppressed incidents. Such rules are alsoflexible and may be tailored to a particular distributed computingsystem and its functions.

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

It will be understood from the foregoing description that modificationsand changes may be made in various embodiments of the present inventionwithout departing from its true spirit. The descriptions in thisspecification are for purposes of illustration only and are not to beconstrued in a limiting sense. The scope of the present invention islimited only by the language of the following claims.

What is claimed is:
 1. An apparatus for determining a number of uniqueincidents in a plurality of incidents for incident processing in adistributed processing system, the apparatus comprising a computerprocessor and a computer memory operatively coupled to the computerprocessor, the computer memory having disposed within it computerprogram instructions that when executed by the computer processor causethe apparatus to carry out the steps of: identifying within theplurality of incidents, by an incident analyzer, attribute combinationentries of location identifications and incident types, wherein eachattribute combination entry has one location identification and a set ofunique incident types corresponding to the location identification;analyzing each location identification in each attribute combinationentry according to a sequence of the attribute combination entriesincluding for each location identification of each attribute combinationentry: determining whether the set of unique incident types within theattribute combination entry includes an incident type not selected forpairing with another location identification in an attribute pair; ifthe set of unique incidents types includes an incident type not selectedfor pairing with another location identification in an attribute pair,selecting the incident type for pairing with the location identificationin an attribute pair and creating an attribute pair of the selectedincident type and the location identification; and counting theattribute pairs; wherein the number of attribute pairs is the number ofunique incidents in the plurality of incidents.
 2. The apparatus ofclaim 1 further comprising computer program instructions that whenexecuted by the computer processor cause the apparatus to carry out thesteps of creating the sequence of the attribute combination entries;wherein creating the sequence of the attribute combination entriesincludes sorting the attribute combination entries based on one of anumber of incident types within each set of unique incident types and anumber of location identifications.
 3. The apparatus of claim 1 whereindetermining whether the set of unique incident types corresponding tothe location identification within the attribute combination entryincludes an incident type not selected for pairing with another locationidentification in an attribute pair further comprises: maintaining alist of selected incident types that have already been selected forpairing with a location identification in an attribute pair; andcomparing the set of unique incident types to the list of selectedincident types.
 4. The apparatus of claim 1 further comprising computerprogram instructions that when executed by the computer processor causethe apparatus to carry out the steps of: identifying attributecombination entries having identical sets of incident types; and mergingthe identified attribute combination entries.
 5. The apparatus of claim1 further comprising computer program instructions that when executed bythe computer processor cause the apparatus to carry out the steps of:determining that the number of unique incidents is greater than or equalto a predetermined number; and executing an action in response todetermining that the number of unique incidents is greater than or equalto the predetermined number.
 6. The apparatus of claim 1 whereinexecuting an action includes at least one of creating an alert andsuppressing an incident.
 7. A computer program product for determining anumber of unique incidents in a plurality of incidents for incidentprocessing in a distributed processing system, the computer programproduct disposed upon a non-transitory computer readable storage medium,the computer program product comprising computer program instructionsthat when executed by a computer cause the computer to carry out thesteps of identifying within the plurality of incidents, by an incidentanalyzer, attribute combination entries of location identifications andincident types, wherein each attribute combination entry has onelocation identification and a set of unique incident types correspondingto the location identification; analyzing each location identificationin each attribute combination entry according to a sequence of theattribute combination entries including for each location identificationof each attribute combination entry: determining whether the set ofunique incident types within the attribute combination entry includes anincident type not selected for pairing with another locationidentification in an attribute pair; if the set of unique incidentstypes includes an incident type not selected for pairing with anotherlocation identification in an attribute pair, selecting the incidenttype for pairing with the location identification in an attribute pairand creating an attribute pair of the selected incident type and thelocation identification; and counting the attribute pairs; wherein thenumber of attribute pairs is the number of unique incidents in theplurality of incidents.
 8. The computer program product of claim 7further comprising computer program instructions that when executed bythe computer cause the computer to carry out the steps of creating thesequence of the attribute combination entries; wherein creating thesequence of the attribute combination entries includes sorting theattribute combination entries based on one of a number of incident typeswithin each set of unique incident types and a number of locationidentifications.
 9. The computer program product of claim 7 whereindetermining whether the set of unique incident types corresponding tothe location identification within the attribute combination entryincludes an incident type not selected for pairing with another locationidentification in an attribute pair further comprises: maintaining alist of selected incident types that have already been selected forpairing with a location identification in an attribute pair; andcomparing the set of unique incident types to the list of selectedincident types.
 10. The computer program product of claim 7 furthercomprising computer program instructions that when executed by thecomputer processor cause the apparatus to carry out the steps of:identifying attribute combination entries having identical sets ofincident types; and merging the identified attribute combinationentries.
 11. The computer program product of claim 7 further comprisingcomputer program instructions that when executed by the computer causethe computer to carry out the steps of: determining that the number ofunique incidents is greater than or equal to a predetermined number; andexecuting an action in response to determining that the number of uniqueincidents is greater than or equal to the predetermined number.
 12. Thecomputer program product of claim 7 wherein executing an action includesat least one of creating an alert and suppressing an incident.